The present invention relates to an image sensing apparatus comprising an electronic anti-shake system that corrects a shake by electrically extracting a sensed image on the basis of shake information of a camera and an image sensing method, and to a shake correction apparatus and method.
Conventionally, image sensing apparatuses such as video cameras and the like have a variety of automatic functions such as AE (auto exposure), AF (auto focus), and the like, and can sense and output a good image.
Also, to compensate for a drop of sensed image quality resulting from a shake of an image sensing apparatus along with recent size reduction of video cameras and higher magnification of optical systems, various shake correction image sensing apparatuses that correct such camera shake have been proposed.
FIG. 1 shows an example of the arrangement of a conventional image sensing apparatus with a shake correction function.
Referring to FIG. 1, this image sensing apparatus is composed of an image sensing unit and a camera shake correction unit. The image sensing unit (camera unit) has a lens 901, image sensing device 902, and signal processing circuit 903. The image sensing device 902 comprises a CCD or the like as a photoelectric conversion means, and converts an optical signal into an electrical signal. The signal processing circuit 903 converts the electrical signal output from the image sensing device 902 into a standard video signal such as an NTSC signal or the like, and has a video output terminal 904 from which the standard video output is obtained.
Also, in FIG. 1, the camera shake correction unit comprises an angular velocity sensor 905 serving as a shake detection means, a DC cut filter 906, an amplifier 907, a computer 908 (to be abbreviated as COM hereinafter) serving as a correction amount calculation means, an electronic shutter 909 serving as an AE control means, a read controller 910, and a timing generator 911.
The angular velocity sensor 905 comprises, e.g., a vibration gyro attached to the image sensing apparatus main body, and detects any shake of the image sensing apparatus main body in two coordinate axes which are fixed with respect to the apparatus main body. That is, the output signal from the sensor 905 contains horizontal and vertical angular velocity signals. In the following description, the output signals from the sensor 905 in the two directions will be simply referred to as an angular velocity signal, and signal processing in the apparatus shown in FIG. 1 is done based on this angular velocity signal, for the sake of simplicity.
The DC cut filter 906 cuts off DC components of the angular velocity signal output from the angular velocity sensor 905, and passes only AC components, i.e., shake components. Note that the DC cut filter 906 may use a high-pass filter (to be abbreviated as an HPF hereinafter) that cuts off signal components in a predetermined frequency band. The amplifier 907 appropriately amplifies the angular velocity signal output from the DC cut filter 906.
The COM 908 has a microcomputer, and includes an A/D converter 908a, high-pass filter (HPF) 908b, integral circuit 908c, D/A converter 908d, and pan/tilt determination circuit 908e. 
The A/D converter 908a converts the angular velocity signal output from the amplifier 907 into a digital signal. The HPF 908b cuts off low-frequency components of the digital output of the A/D converter 908a, and has a function capable of varying its characteristics in an arbitrary frequency band. The integral circuit 908c integrates the output (angular velocity signal) from the HPF 908b to output an angular displacement signal, and has a function capable of varying its characteristics in an arbitrary frequency band. The DA converter 908d converts the integral signal output from the integral circuit 908c, i.e., the angular displacement signal, into an analog signal. The pan/tilt determination circuit 908e determines panning/tilting on the basis of a digital angular velocity signal 908f and an integral signal 908g (i.e., angular displacement signal) obtained by integrating the angular velocity signal, from which the low-frequency components have been cut, by the integral circuit 908c. The pan/tilt determination circuit 908e performs panning control (to be described later) on the basis of the levels of the input angular velocity signal 908f and angular displacement signal 908g. Also, the obtained angular displacement signal 908g is used as a target value for shake correction in control done at a later time.
The electronic shutter 909 especially controls the charge storage time period in the image sensing device 902 in this embodiment. Although not shown, the electronic shutter 909 operates in accordance with the exposure state of the image sensing device 902 or user's operation.
The shake correction control in FIG. 1 controls the read start position of the image sensing device 902 on the basis of the displacement amount signal 908g that represents shake amount. For this purpose, the read controller 910 moves the read start position of the image sensing device 902 in correspondence with the angular displacement signal 908g. The read controller 910 also controls the storage time period of the image sensing device 902 by the electronic shutter 909 at the same time.
The timing generator 911 generates drive pulses for signal storage or read of the image sensing device 902 on the basis of control information supplied from the read controller 910, and sends them to the image sensing device 902.
The operation of the pan/tilt determination circuit 903e in the COM 908 will be described in detail below.
The determination circuit 908e receives the angular velocity signal 908f and angular displacement signal 908g, and determines that panning or tilting is underway in place of a shake, when it is determined that the angular velocity signal 908f is equal to or higher than a predetermined threshold value or when the angular velocity signal 908f is equal to or smaller than the predetermined threshold value but the angular displacement signal 908g is equal to or higher than a predetermined threshold value. If it is determined that panning or tilting is underway, the determination circuit 908e shifts the cutoff frequency range of the HPF 908b to the higher-frequency side to change the characteristics so that the shake correction system does not respond to lower frequencies. Furthermore, the determination circuit 908e executes so-called “panning control”. In this panning control, the time constant of the integral characteristics of the integral circuit 908c is shifted in a direction to become shorter so as to move the central position of the image extraction position (1203 or 1204 in FIG. 5; to be described later) by the read controller 910 to the center of the entire area of the image sensing device 902, thereby bringing a value stored in the integral circuit 908c close to a reference value (a value that can be assured while no shake is detected). During this panning control, the angular velocity signal 908f and angular displacement signal 908g are detected. When the panning/tiling determination result changes to NO, i.e., when the user has finished panning or tilting, after the determination circuit 908e lowers the cutoff frequency to restore the original value and to broaden the shake correction range, it leaves the panning control.
Pan determination will be explained below with reference to the flow chart in FIG. 2. Note that the processing shown in this flow chart repeats itself at predetermined timings.
In step S1001, the angular velocity signal amplified by the amplifier 907 is converted from an analog amount into a digital value that can be processed inside the COM 908. In step S1002, the HPF 908b executes HPF processing using a cutoff frequency fc prepared in the previous processing. In step S1003, the integral circuit 908c integrates using a time constant τ prepared in the previous processing. In step S1004, the D/A converter 908d converts the integral result (i.e., the angular displacement signal 908g) in step S1003 into an analog amount, and outputs it.
It is then checked in step S1005 if the angular velocity signal 908f is equal to or higher than a predetermined threshold value TH1. If the angular velocity signal is equal to or higher than the predetermined threshold value TH1, the value of the cutoff frequency fc to be used in the next HPF calculation is set to be higher by a predetermined value Δ1 than the current value in step S1006. The shift operation of the cutoff frequency to the higher-frequency side sets the total attenuation factor of a signal in a low-frequency range to be higher than the current one. In step S1007, the time constant τ used in the integral calculation is set to be shorter by a predetermined value than the current value, so that the angular displacement output 908g comes close to the reference value.
On the other hand, if it is determined in step S1005 that the angular velocity signal 908f is not equal to or higher than the predetermined threshold value TH1, it is checked in step S1008 if the integral value 908g is equal to or higher than a predetermined threshold value TH2. If the integral value 908g is equal to or higher than the predetermined threshold value TH2, the flow advances to step S1006; otherwise, the flow advances to step S1009. In step S1009, the value of the cutoff frequency fc to be used in the next HPF calculation is set to be lower by the predetermined value Δ1 than the current value, so as to set the signal attenuation factor in the low-frequency range to be lower than the current one. In step S1010, the time constant τ to be used in the integral calculation is set to be longer by a predetermined value than the current value to improve the integral effect, and this processing ends.
With the above-mentioned control, when the shake velocity (angular velocity) value is large or when the integral value may be saturated, a higher cutoff frequency fc is set (step S1006) or a shorter integral time constant τ is set (step S1007), thus preventing the output from the integral circuit 908c from being saturated. As described above, since this integral value is D/A-converted, and is used in shake correction (to be described later) by the read controller 910, preventing the integral circuit 908 from being saturated leads to preventing the correction target value for shake correction from being saturated, realizing stable anti-shake control.
The shake correction in this prior art will be briefly described below with reference to FIG. 3. This shake correction is implemented by changing the read position of image data form the image sensing device 902, that is, changing the image extraction position, in accordance with the output from the integral circuit 908c. 
Referring to FIG. 3, reference numeral 1100 denotes an entire image sensing area of the image sensing device 902. Reference numeral 1101 denotes an extracted image area, which is extracted from the entire image sensing area 1100, and is actually converted into a standard video signal to be output as a video signal. Reference numeral 1106 denotes an object which is sensed by the photographer. In the following description, the outer sides of the extracted image area will be called an extraction frame.
When the extracted image area 1101 is converted into a standard video signal and the converted signal is displayed, a video denoted by 1105 is obtained. That is, in FIG. 3, reference numeral 1105 denotes a video area of a monitor that reproduces a video signal; and 1106′, an object (principal object) reproduced on the video area 1105 of the monitor. By outputting a portion obtained by excluding the peripheral portion from the entire image sensing area 1100 of the image sensing device 902 by sensed image extraction (to be described later), the video area 1105 on the monitor can be reproduced.
Furthermore, in FIG. 3, reference numeral 2000 denotes a change in image when the photographer who senses the object 1106 has inadvertently swayed the image sensing apparatus in the lower left direction indicated by arrows 1102, 1102′, and 1102″. When a shake in the lower left direction has taken place, the object 1106 moves in the upper right direction indicated by an arrow 1104 on the entire image sensing area 1100. In this state, if extraction is done using a frame area 1101′ located at the same position as that of the above-mentioned extraction frame 1101, a video signal that represents an image in which the object 1106 has moved by a vector amount indicated by an arrow 1104 is generated.
When the extraction frame is moved from the position 1101′ to a position 1101″ using a displacement amount 1103 (i.e., a shake correction target value) obtained based on the shake amount of the image sensing apparatus, and the image is extracted using this displaced extraction frame 1101″, the same video as the video 1105 can be obtained. Using this principle, shake correction of an image is implemented.
Displacement control of the extraction position of the image sensing area will be explained below with reference to FIG. 4.
Referring to FIG. 4, reference numeral 1200 denotes the entire pixel set of the image sensing device 902. Reference numeral 1201 denotes a unit pixel which corresponds to one photoelectric conversion element. Based on electrical drive pulses generated by the timing generator 911, storage and read are controlled in units of pixels.
Reference numerals 1202 and 1203 denote extraction frames which are the same as the extraction frame 1101 shown in FIG. 3, i.e., those when no shake is produced. A case will be exemplified below wherein a video signal is extracted using the extraction frame 1202 shown in FIG. 4.
Initially, photoelectrically converted charge amounts are read out in turn from pixel “S” in a direction indicated by an arrow 1205 (i.e., horizontal direction). This read is started in synchronism with the synchronization period of the output video signal. Since pixel data corresponding to the synchronization period is not displayed, that pixel is read out at a rate higher than the data read rate of a pixel to be displayed, i.e., the normal read rate. The first synchronization period continues to a position one pixel before pixel “A” in FIG. 4.
During an actual video period after the end of the first synchronization period, charges in pixels A to F are read out as image information for one line of a video signal according to the normal read rate.
Furthermore, during the horizontal synchronization period up to the next line, all pixels from a pixel next to pixel F to a pixel before pixel G are read out at a transfer rate higher than the normal read rate to prepare for a read of the next actual video period. A read from pixel G is then started as in that from pixels A to F.
By controlling the read timings, as described above, all pixel data bounded by the extraction frame 1202 can be selectively extracted from the entire image sensing area 1100 of the image sensing device 902 and can be converted into a video signal.
Shift of the extraction position when a sensed image has moved upon movement of the image sensing apparatus resulting from a shake will be explained below. When it is detected that the object has moved (=the image sensing apparatus has been shaken) by an amount indicated by an arrow 1204, the extraction frame is changed from the position 1202 to a position 1203. By extracting an image using the changed extraction frame, a video free from any movement errors of the object can be obtained.
In order to change the extraction position, when the read start position is shifted from pixel “A” to pixel “B”, an objective partial image can be selectively extracted from the image sensing area 1100 and can be converted into a video signal as in the read starting from pixel “A”.
In practice, photoelectrically converted charge amounts are read out in turn from pixel “S” in the direction indicated by the arrow 1205 as in the read of the contents of the extraction frame 1202. This read is started within the synchronization period of the output video signal, and all pixels up to a pixel one pixel before pixel “B” are read out at a transfer rate higher than the normal read rate. During the actual video period, a read can start from pixel “B”.
To restate, the displacement vector 1204 is generated by the read controller 910 on the basis of the displacement amount signal 908g. More specifically, the synchronization period shown in FIG. 4 is determined by the read controller 910 on the basis of the displacement amount signal 908g. In this fashion, in image extraction the peripheral image sensing area of the image sensing device 902 is partially read during the synchronization signal period that falls outside the actual video period by an amount corresponding to shake correction information, and only pixel data extracted in the actual video period are selectively read out, thus obtaining a video signal free from an image blur arising from the shake of the image sensing apparatus.
As can be seen from FIG. 1, the correction amount (integral value 908g) is calculated based on the output from the angular velocity sensor 905 independently of storage/read of image data in the CCD 902. Since shake correction that has been explained in relation to FIG. 3 extracts image data stored in the photoelectric conversion elements of the image sensing device 902 using an appropriate extraction frame, no problem is posed even when the shake of the camera main body is large and the value of the integral output 908g increases, unless the electronic shutter 909 operates. This is because shake correction based on the output from the COM 908 controls the image extraction position and does not influence the charge storage time period.
The reason why shake correction appropriately effects unless the electronic shutter 909 operates for exposure amount control will be explained below with reference to FIG. 9.
FIG. 5 is a timing chart showing movement of an image stored in the image sensing device 902 and the extraction timings along with an elapse of time.
Referring to FIG. 5, reference numeral 1301 denotes a moving amount of an image on the surface of the image sensing device 902 upon occurrence of shake of the image sensing apparatus, in other words, shake of the image sensing apparatus itself. In the example shown in FIG. 5, the image sensing apparatus has swayed to first increase the image moving amount and then to decrease it.
Referring to FIG. 5, reference numeral 1314 denotes a vertical synchronization signal (V synchronization signal) generated in the image sensing apparatus. More specifically, when this V synchronization signal is low level, known vertical synchronization control is done. The duration between two consecutive V synchronization signals corresponds to one field time period. More specifically, during the high-level period of the V synchronization signal, image data photoelectrically converted during this field (odd field) in the CCD 902 are stored parallel to a read of image data stored one field (even field) before.
Reference numerals 1315, 1316, 1317, and 1318 denote the sampling timings of the A/D converter 908a; and 1319, 1320, 1321, and 1322, the output timings of a shake correction target value (i.e., displacement vector 1204) from the COM 908 serving as the correction amount calculation means to the read controller 910. Reference numerals 1323, 1324, 1325, and 1326 denote timings at which the read controller 910 uses the correction target value output from the COM 908 in the read control of the image sensing device 902. Since the timings 1323, 1324, 1325, and 1326 match the low-level timings of the V synchronization signal, image extraction control is started from the synchronization period for a field (even field) next to the field (e.g., odd field) in which motion has been detected.
Reference numeral 1306 in FIG. 5 denotes a change (a displacement per unit time period) in moving amount of an image during one field. When the electronic shutter 909 is inoperative, the whole period between the two vertical synchronization periods corresponds to the storage time period of the image sensing device 902. Hence, the change 1306 represents the displacement of an image on the surface of the image sensing device 902 during the storage time period in one field. When the image displacement amount 1306 is integrated across the synchronization period (the period between the two V synchronization signals), the integral value indicates the barycentric position of an image during a unit storage period. Assume that the barycentric positions of displacement amounts 1306, 1307, 1308, and 1309 have been detected at positions 1310, 1311, 1312, and 1313. The barycentric positions 1310, 1311, 1312, and 1313 of an image assume a value close to a reference value 1327 indicated by the broken line as long as the maximum value of the slope of the displacement amount 1301 is small to some extent, although they vary depending on the slopes and curvatures of the image displacement amounts 1306, 1307, 1308, and 1309 per unit time period. In other words, while the electronic shutter is inoperative, the differences between the reference value 1327 and the values of the barycentric positions 1310, 1311, 1312, and 1313 as the integral values of the displacement amounts 1306, 1307, 1308, and 1309 become small.
The central time of one period of the V synchronization signal (i.e., central times 1302, 1303, 1304, and 1305 of video periods) is set as the sampling timing of the angular displacement signal (the output from the D/A converter 908d) used in shake correction. This timing is indicated by “sampling timings” 1315, 1316, 1317, and 1318 in FIG. 5. Unless the electronic shutter operates, since the CCD 902 records the displacements 1306, 1307, 1308, and 1309 for one frame period, the central time positions of the displacement amounts 1306, 1307, 1308, and 1309 nearly match the sampling timings 1315, 1316, 1317, and 1318. Since the read control of the image sensing device 902 is done based on such timing, the read controller 910 executes shake correction control to center on the image stored at the sampling timing 1315.
When the electronic shutter is inoperative, a video signal stored during the period of the image displacement amount 1306, 1307, 1308, or 1309 on the surface of the image sensing device 902 for the storage time period in one field is output from the image sensing device 902 while the next field is being stored. If the previously determined correction target value data is used at the time of this read, no time mismatch occurs between the shake which is being stored and the time period required until the correction target value.
Furthermore, in FIG. 5, reference numeral 1327 denotes the reference value (=center at the time of correction, correction center) upon correction, and control is done so that the integral value of an image stored at the sampling timing 1315, 1316, 1317, or 1318 of the shake information matches this reference value 1327.
In this way, since the shake information sampling point is set at the center 1302, 1303, 1304, or 1305 of the synchronization period, the timing the value of the displacement amount 1306, 1307, 1308, or 1309 matches the reference value 1327, i.e., the time of the intersection between the displacement amount curve 1306, 1307, 1308, or 1309 and line 1310, 1311, 1312, or 1313 is set at a point in the neighborhood of the shake center of an image during the storage time period of the image sensing device 902, thus achieving shake correction by extraction.
In the system for correcting (or compensating for) an image blur shown in FIGS. 1 to 5, since the timing of determination and correction of the shake correction amount based on shake correction information is reached once per read period of the image sensing device 902, detection of shake information and calculation of the correction target value used in shake correction are made once per read.
More specifically, as has been explained with reference to FIG. 5, the shake information (angular velocity) is sampled once per synchronization period (one field period) at the central time 1315, 1316, 1317, or 1318, and the correction target value to be output based on the sampling data is formed once per synchronization period for one field, as indicated by 1319 to 1322. Of course, the read timing control of the image sensing device 902 is also determined at the beginning of the vertical synchronization period and remains unchanged until the next timing.
In this conventional image sensing apparatus, detection of the image blur amount and calculation of the target value of a correction control variable are made at nearly the central time (1315 to 1318) of the synchronization period. In such image sensing apparatus, the electronic shutter 909 operates for the purpose of controlling the exposure amount, temporarily resets charges to zero during the storage time period of the image sensing device 902, and restarts storage, thus controlling the storage time period of the image sensing device 902. When the electronic shutter is operated, the barycentric position of an image during the actual storage time period and the reference position upon correction may have a large difference unlike in the case described with reference to FIG. 5.
Why such large difference is produced will be explained below with reference to the timing chart shown in FIG. 6.
FIG. 6 shows the movement of an image stored in the image sensing device 902 and the extraction timings along with an elapse of time, as in FIG. 5.
Referring to FIG. 6, reference numeral 1301 denotes the moving amount of an image on the surface of the image sensing device 902 as in FIG. 5, in other words, the shake of the image sensing apparatus itself.
Also, points 1340, 1341, 1342, and 1343 respectively indicate the refresh timings at which the charges on the image sensing device 902 are reset. In the example shown in FIG. 6, such timing is set at nearly the center in one field, i.e., the refresh operation continues until a timing (timings 1340, 1341, 1342, and 1343) near the central time of one field period. Hence, if charges are reset at that timing, the electronic shutter speed assumes a value half the shutter speed in normal storage (FIG. 5). At this time, time periods indicated by solid curves 1344, 1345, 1346, and 1347 indicate movements of an image effectively stored as a sensed image in the image sensing device 902.
Reference numeral 1314 denotes a vertical synchronization signal generated in the image sensing apparatus as in FIG. 5. Reference numeral 1328 denotes a change in moving amount of the image effectively stored as the sensed image during one field, i.e., the displacement amount per unit time period. Reference numerals 1328, 1329, 1330, and 1331 denote displacements of an image on the surface of the image sensing device 902 during the effective storage time periods for one field. These displacements continue for periods nearly half the image displacement amounts 1306, 1307, 1308, and 1309 described above with reference to FIG. 5, and account for roughly half the entire displacement amounts.
When central times 1302 to 1305 between the synchronization periods of the synchronization signal 1314 are used as the sampling timings of an angular displacement signal used in shake correction as in the description of the prior art, the storage start time of each of the displacement amounts 1328 to 1331 per unit time period is set as a correction point upon image extraction, and shake correction control is done based on the displacement amount at that timing to locate an image stored at the sampling timing at the correction center at its read timing from the image sensing device 902.
Reference numeral 1327 denotes a reference position upon correction, and control is made to locate an image stored at the timing 1302 as a sampling point on this reference position 1327. Extraction control based on displacement amounts 1329 to 1331 per unit time period is similarly made.
A line 1332 indicates the integral value of the displacement amount 1328 per unit time period, i.e., the barycentric position of an image during a unit storage time period as in the above description. As can be seen from the barycentric position 1332 of the image, the difference from the reference position 1327 upon correction becomes larger than that in the prior art described above with reference to FIG. 5. The same applies to barycentric positions 1333 to 1335.
When storage & read control of the image sensing device 902 by the electronic shutter operation is made, the target correction reference position separates from the actual image barycentric position upon shake correction, and a video cannot be obtained with a sufficient anti-shake effect.
Furthermore, as the storage time period becomes shorter in relation to the electronic shutter operation, this difference becomes larger. Hence, the problem that the correction reference position separates from the image barycentric position cannot be ignored.
The first problem in the prior art has been explained.
As the second problem in the prior art, correction errors are produced in shake correction due to errors in the response characteristics of the shake detection means with respect to shake frequency. More specifically, such correction errors are produced when the response characteristics of an angular velocity sensor such as a gyro sensor or the like used as shake detection means with respect to frequency worsens as the frequency becomes higher.
FIG. 7A shows the typical frequency response characteristics of an angular velocity sensor. In FIG. 7A, the abscissa plots the frequency ranging from about 1 Hz to 20 Hz, which is considered as the camera shake frequency range of an image sensing apparatus. The ordinate plots the phase characteristics, which represent a lead or delay of the output from the angular velocity sensor with respect to the actual shake.
When the shake frequency applied is low, the gyro sensor 905 exhibits characteristics indicated by a curve 292 in FIG. 7A, which are free from any phase delay or lead. As the frequency applied to the sensor becomes higher, the response characteristics of the sensor 905 show a phase delay.
A gyro sensor which is popularly used, e.g., a vibration gyro sensor produces a delay around 10 deg at 20 Hz.
Since this delay amount of the sensor output as the shake detection means finally appears as correction errors upon shake correction, if the delay amount of the response characteristics of the shake detection means increases, the correction performance of shake correction (anti-shake rate) deteriorates.
The second problem resulting from a response delay unavoidable in a shake detection sensor such as a gyro sensor or the like has been explained.
The first and second problems occur when the angular sensor signal is sampled once per field period.
A problem (third problem) that occurs in a prior art which samples a sensor signal a plurality of number of times within one field period will be explained below.
In a conventional image sensing apparatus with a camera shake prevention function, an angular velocity sensor such as a gyro or the like is used to detect a shake signal as in the above-mentioned prior art, and the output from this sensor is input via an A/D converter. At this time, in order to improve the precision of data output from the sensor and to prevent noise, a sensor output is sampled a plurality of number of times within one field period, the obtained outputs are A/D-converted into a plurality of digital data, and the digital data are integrated.
Since the hardware arrangement of this conventional multi-sampling image sensing apparatus is substantially the same as that shown in FIG. 1, FIG. 1 will be quoted.
FIG. 7B is a timing chart for explaining the “sampling timings” of an angular velocity sensor 905 used in this conventional multi-sampling image sensing apparatus, the integral value calculation timings of an integral circuit 908c, and the “correction timings” at which shake correction is done based on the obtained integral value.
Referring to FIG. 7B, reference numerals 801 to 803 denote vertical synchronization signals; and 1001 to 1008, sampling timings, i.e., the conversion timings of an angular velocity signal obtained from the angular velocity sensor 905 into a digital amount by an A/D converter 908a. These sampling timings have an appropriate phase relationship with the vertical synchronization signals 801 to 803, and intervals between neighboring timings are equal to each other, as shown in FIG. 7B.
In the conventional multi-sampling image sensing apparatus, angular velocity information sampled at each of the timings 1001 to 1008 by the A/D converter 908a is integrated by the integral circuit 908c at each of timings 1101 to 1108. In the example shown in FIG. 7B, integral results are obtained at timings 1104 and 1108. A read controller 910 receives these integral results (shake correction target values) at timings 1301 and 1302 to execute camera shake correction.
However, in this conventional multi-sampling image sensing apparatus, when shake correction control is continuously done at the above-mentioned control timings, since the shake correction target values are calculated at fixed timings having equal time intervals, the use efficiency of a COM is poor.
More specifically, as shown in FIG. 7B, shake information is sampled at the sampling timings 1001 to 1008, and a correction amount calculation is started at timings 1101 to 1108 in correspondence with the sampling timings. These calculation time periods are indicated by processing timings 1201 to 1208 in the COM. In addition, the calculation time periods required for the read controller 910 to calculate extraction points after correction data is determined during the synchronization period are indicated by 1401 and 1402.
Shake information sampling can be done by executing interrupt processing using an internal timer of the COM with reference to vertical synchronization signals, and the time period required for such sampling is very shorter than those required for other calculations and the like. The correction amount is discretely calculated in synchronism with the sampling timings of the angular velocity information, and the use efficiency of the COM itself is poor. Such idle state of the COM frequently occurs if a large number of times of sampling of shake information is set.
More specifically, in case of, e.g., a standard video signal such as an NTSC signal or the like, sampling must be done around five to 10 times per field. For example, when a shake signal (angular velocity signal) is sampled 10 times per field, if about 16.7 ms as one field time period are equally divided into 10 periods, one sampling period is around 1.67 ms. The time period (duration 1201) required for calculations made in the COM falls within the range from about 0.3 to 0.8 ms although it largely depends on the specifications of the computer used. Hence, the ratio of the running time period of the computer to the period=1.67 ms between neighboring sampling timings falls within the range from about 18% to 48%, and the remaining time period which accounts for more than 50% of the whole time is an idle period.
Even when this idle period is used for another application, if that application requires continuous processing, such idle period is not suitable for the application since one field has already been divided into a plurality of periods. If calculation processing such as auto-focus control that occupies a continuous time period and has higher priority is done, the correction amount calculation timing cannot be completed before the next sampling timing, and consequently, one sampling data is lost, i.e., effective correction data cannot be obtained.